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Rapid advancement characterizes U. S. high-technology 
industries, and the need for technically qualified employees 
continues to increase (Riggs, 2000). The increased need for 
technology education is verified hy recent research that indicates 
graduates of university technology programs are hired 
immediately after graduation (Nock & Shults, 2001). To maintain 
a strong technical workforce, there is an increasing need for 
colleges and training institutes to update their curriculum so that 
it addresses the latest technologies and industry practices 
(Frenzel, 2003). The National Science Foundation will provide an 
estimated $40,000,000 in educational grant funding for projects 
that team with industry for this purpose (NSF, 2003). Technical 
subject matter experts will play an essential role in helping 
achieve this goal, hut their role differs from that of the 
instructional developer (Lee, 1994). While the technical expert 
provides what becomes the course content material, it is the 
instructional developer’s task to produce instructional 
components (e.g., lectures, practice, tests) that facilitate and 
verify the acquisition of the target knowledge (Dick & Carey, 
1996). However, development of curriculum is time-consuming, 
challenging, and costly, therefore selecting a subject matter 
expert (SME, pronounced “smee”) who is capable and suitable for 
the job is critical. 

In this paper, the process of obtaining technical 
information from a SME is referred to as “knowledge extraction.” 
For each curriculum project, the instructional developer must 
conduct the knowledge extraction process with at least one SME. 
Considering the increasing volume of technical curriculum 
needed, the cost of its development, and the critical role of SMEs 
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in this endeavor, it is important to select SMEs who are 
technically qualified. Yet, is technical knowledge all that is 
important when choosing a SME to support curriculum 
development? This study argues that technical expertise is 
essential hut not sufficient for providing the hest support. The 
problem investigated here is the identification of a SME’s 
personal and professional qualities that will provide the 
instructional developer the most efficient and effective support to 
the knowledge extraction process and subsequent revision and 
finalizing of the technical material. 

Capability of Subject Matter Experts 

SMEs need to possess certain capabilities to effectively 
communicate and help organize their technical knowledge in a 
format that is conducive to curriculum production. SME abilities 
are referred to as “capability factors.” The capability represented 
by SMEs’ breadth and depth of knowledge refers to their level of 
understanding of the topic area and its associated application and 
practice in the workplace. Breadth and depth of knowledge do not 
refer to the ability to explain the subject matter. Highly 
knowledgeable people often have difficulty articulating and 
explaining their technical knowledge to others who lack a 
technical background (Gayeski, Wood, & Eord, 1992; Gordon & 
Gill, 1997). Eor this reason, industry experience and articulation 
skills were proposed as important capability factors along with 
depth of knowledge and breadth of knowledge. When combined 
with industry experience, articulation skills enable a SME to 
clearly describe what needs to be taught and can verify and 
validate content based on actual experience. A fifth capability 
factor, teaching experience, is proposed on the assumption that a 
SME who understands instructional practice can suggest effective 
ways to present the material to learners. 

Technical Expertise: The Industry SME 

Because companies strive to implement the most 
advanced technologies and best practices in order to be 
competitive, currently employed technicians and engineers are 
often a good source of up-to-date technical knowledge. 
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Technicians, whose johs focus on specific tasks and processes, 
tend to develop in-depth and detailed knowledge in a particular 
domain or “domain-specific knowledge.” Alexander and Judy 
(1988) stressed the value of someone with domain-specific 
knowledge for providing expertise in solving problems and 
performing complex tasks. However, SMEs whose performance 
and joh duties lie within a narrow scope can lack the breadth of 
knowledge which may be needed to provide students enrolled in 
college technology programs with both specific knowledge and a 
general understanding of a technology field. Breadth and scope 
are necessary for curriculum which is designed to impart both 
specific knowledge and a broad understanding of the field. 
Industry SMEs who hold management positions often have a 
wider breadth of knowledge than technicians and engineers, but 
acquiring their time to support curriculum development can be 
more difficult. 

When searching for an industry SME, instructional 
developers should first consult with the SME’s employer. 
Managers can identify technical personnel with specific 
knowledge sets, and they may be able to point out SMEs who are 
most qualified and available to help with curriculum 
development. It may, however, be difficult to get permission to 
work with the most knowledgeable experts because their time is 
highly valued by the company. Seeking a manager’s approval and 
advice in locating potential SMEs demonstrates professional 
courtesy and promotes goodwill which can leverage assistance 
with future development projects. Developing a good relationship 
with a local company can pave the way to current and future 
sources of technical SMEs. In addition, other cooperative 
agreements such as collaborative training agreements, recruiting 
privileges, and joint workforce programs can garner curriculum- 
development support from industry. 

Pedagogical Expertise: The Faculty SME 

Industry and faculty SMEs may possess different types of 
expertise, both of which can be uniquely valuable to supporting 
the curriculum development process. While industry SMEs’ 
knowledge tends to exhibit a highly focused perspective on the 
target domain and the specific skills necessary to perform a job, a 
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faculty instructor’s knowledge has a broader focus. Faculty 
promote a macro perspective for students of technology with a 
balance of emphasis on depth and breadth of knowledge. This is 
necessary to enable students to make career choices and to build 
their potential for a range of occupations in the field. Faculty also 
possess skill in articulating complex ideas and applying 
pedagogical methods that help students learn. Good teachers can 
explain difficult concepts using analogies and concrete examples, 
and they are able to spot particular areas within technical topics 
where students may stumble and require additional guidance. 
Faculty can help an instructional developer shape the curriculum 
and learning activities so that domain-specific information is tied 
to the larger and broader body of knowledge and is linked to core 
disciplines such as math, physics, and chemistry. Yet faculty who 
do not have recent industry experience may not be able to supply 
as up-to-date and in-depth information as industry SMEs. This is 
especially true of an area such as electronics which can change 
significantly in a six- month time period. 

When seeking a faculty SME, department chairs, deans, 
and peer teachers can help identify the best candidates. As in 
industry, professional courtesy will help develop good 
relationships that can be leveraged to support future projects. 
Seeking permission from administrators prior to asking for an 
instructor’s support smoothes the collaboration. 

Suitability of Subject Matter Experts 

Besides possessing depth and breadth of technical 
knowledge and being able to articulate and communicate 
effectively, the SME should be readily available to support the 
curriculum development project, possess sufficient interpersonal 
skills to work effectively with others, have a positive attitude 
toward the project goals and development team, and be affordable 
relative to the project budget. 

Availability 

The SME must have sufficient time to devote to the 
curriculum development process. Even a small body of technical 
curriculum requires significant time to produce. The SME’s 
availability is especially important in the first stage of 
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development since learning objectives and other critical 
instructional components are based directly on the SME’s 
technical content knowledge. Additionally, the SME is usually 
needed to help revise the curriculum after it undergoes a tryout 
and evaluation period with students and instructors. 

Availability can be a challenge because curriculum 
development is rarely part of a SME’s normal job responsibilities. 
Short but intensive work sessions of two to three hours with 
substantial one-or-two-day breaks is often most convenient for an 
SME who holds a full-time job. Short sessions also reduce the 
mental and physical stress associated with knowledge extraction. 
In addition, the breaks provide time for the SME to reflect on the 
knowledge extraction sessions and may promote recall of 
important information and trigger ideas on how to structure the 
course content. The time can also be utilized by the instructional 
developer to prepare drafts for review by the SME in the next 
session. SMEs who cannot commit their availability to such a 
schedule represent a risk to the success of the curriculum project. 

Time commitments should be specified in a legal contract 
or agreement before work on the curriculum begins. Eormal 
contracts yield both legal and psychological value. They state 
time and task commitments in specific terms that reduce the 
potential for inaccurate expectations. 

Interpersonal Skills 

If the SME is unable to “get along” or communicate 
effectively with the instructional developer or project team 
members, it will be difficult or even impossible to generate an 
acceptable curriculum. To assess interpersonal skills, the 
prospective SME should be introduced to project team members 
and be closely observed for communication style, mannerisms, 
sense of humor, level of comfort, and professional courtesy. An 
SME with experience working in teams is more likely to have 
developed the interpersonal skills that promote effective 
teamwork and the ability to manage conflicts or disagreements 
with others. Such skills may be inferred from a SME’s history of 
past teamwork experiences, or a discussion with the SME’s 
supervisor or coworkers can also provide general information 
about his or her team performance and interpersonal skills. In 
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these discussions inquiries should he restricted to general topics. 
Questions of a highly personal nature should he avoided. 

Attitude 

A SME’s attitude toward the educational product, the 
instructional developer, and the development team can have a 
strong effect on his or her performance. If a SME believes the end 
product will he of little value, he or she will expend less effort in 
developing a quality product. Conversely, a SME with a positive 
attitude toward the product is likely to have an elevated personal 
interest and a sense of pride in the project and pay meticulous 
attention to the accuracy and organization of the technical 
information. Attitude can he assessed, to some degree, hy asking 
questions about the SME’s perceived value of the target 
curriculum. Since most attitudes are based on deep-rooted beliefs 
that develop over a lifetime of experience and are not easily 
changed (Petty & Cacioppo, 1981), expecting a negative attitude 
to change during the curriculum development process is not 
realistic. Asking open-ended questions such as “How do you think 
the quality of curriculum determines how well technical people 
perform?” or “What kind of instruction do you think promotes the 
strongest technical knowledge?” can help assess the SME’s 
attitude toward the product. More information about the SME’s 
attitude can emerge during discussions with the SME about the 
project and its benefits to its intended users. 

Cost 

Although the availability of funds will vary greatly across 
organizations and specific curriculum projects, cost is always a 
consideration. Cost is accrued almost exclusively from labor hours 
on curriculum development projects. While some SMEs require 
significant compensation for their time, others may volunteer 
their services or be compensated by funds acquired outside of the 
project budget. Considering the capability and suitability factors 
already discussed, a SME should not be chosen based on cost 
alone. If a SME’s knowledge and ability is inadequate, knowledge 
extraction and development of technical content will take longer 
and cost more, even if the SME agrees to work at a low 
compensation rate. 
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When direct compensation for SME support is required, 
two basic types of contracts can be considered: fixed-fee and 
hourly rate. Fixed-fee contracts pay a specified amount to the 
SME for clearly defined “deliverables.” The volume of material 
and consulting time is not always spelled out or even implied in a 
fixed-fee contract, so this compensation method can be risky to 
both parties unless very explicit definitions of the deliverables 
(e.g., specified volume of content on a well-defined topic) are 
possible. Even when a deliverable is specified, the time required 
and the quality of the end product are hard to regulate. Fixed-fee 
contracts work best for projects in which time, resources, and 
degree of effort can be accurately estimated. 

A contract that compensates by an hourly pay rate is 
often preferable for greater flexibility in adjusting duration of 
SME support and because a fixed fee is not always acceptable to 
the SME. On the downside, paying by the hour may reduce the 
SME’s motivation to work efficiently on the project. Also, staying 
within the limitations of the project budget is more difficult if a 
maximum number of hours is not specified in advance. 

Recruiting recent industry retirees or graduate students 
may reduce the cost of SME support. These SMEs can contribute 
two different types of knowledge. Industry retirees are likely to 
have robust knowledge relevant to the workplace including 
business culture and industry practices and may be willing to 
volunteer their time. Graduate students may lack knowledge of 
industry practices but may possess more up-to-date technical 
knowledge than the retiree. A graduate student SME may also be 
more willing to work at lower pay rates than a SME who is 
employed full-time. 

Industry SME support can also sometimes be acquired 
without direct compensation. Large companies have workforce 
development funds to support education and training. Some 
SMEs can acquire release time from their normal job duties via 
compensation from these funds. In such cases, SME support is 
obtainable at no cost to the developer or through a cost-sharing 
agreement. Such a cost reduction is possible particularly if a 
SME’s employer views the curriculum as valuable to their 
workforce needs. 
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Instructional Developers’ Survey and Results 

The capability and suitability factors proposed here were 
initially based on instructional design theory and the author’s 
experience with SMEs. Additional validity was established by an 
on-line survey of instructional developers. Seventeen 
instructional developers who had experience working with SMEs 
were recruited to respond to the survey. The population of 
respondents represented instructional development efforts within 
academe, the military, and industry. 

Instructional Developers’ Survey Design 

An HTML— based survey was created to collect responses 
via the Internet. HTML, or Hypertext Markup Language, enabled 
the survey to be posted on the World Wide Web in an interactive 
format. The curriculum developers’ responses were automatically 
recorded in a database and later downloaded for analyses. Online 
directions explained the purpose of the survey and how to respond 
to each of its three parts — (1) demographics, (2) SME capability 
and suitability factor ratings, and (3) free-responses. The 
demographic questions inquired about the respondent’s type of 
employment organization, occupation, and years of experience. 
Choices provided for employment organization were college or 
university, military, manufacturing, and other. Choices for 
occupation were curriculum developer, training developer, and 
other. A numerical field was provided for respondents to enter 
their years of experience. In the second section, respondents 
ranked their rating for each SME capability and suitability factor 
on a five-choice scale, ranging from 0 to 4, in which 0 indicated no 
importance and 4 indicated high importance. The third, free- 
response, section was provided for respondents to enter other 
factors that they felt were important characteristics of a SME but 
which were not included in the factor rating area of the survey. 

Method and Results 

Curriculum development experience of the respondents 
ranged from one to 33 years {M = 14.8, Mdn = 13.0). Participants’ 
employment organizations were identified as follows: nine, college 
or university; four, military; two, manufacturing; and two, other. 
Although the purpose of the survey was to examine respondents’ 
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ratings of the SME capability and suitability factors, statistical 
tests were first conducted to determine if any differences existed 
as a function of the respondents’ demographics. This was 
accomplished via Statistical Package for the Social Sciences 
(SPSS) Exact Tests because the respondents were not randomly 
sampled, and the data could not be assumed to be normally 
distributed (Mehta & Patel, 1996). All tests employed a standard 
significance level (p < .05). Eisher’s Exact Test revealed no 
significant effects for employment organization or occupation, nor 
was there a significant correlation between years of experience 
and ratings of importance on each of the nine SME factors. There 
was also no significant correlation among factors. 

Rating results on factors are shown in Table 1. Results 
are described in terms of each factor because “capability” and 
“suitability” were used for organizing the discussion rather than 
identifying distinct categories. Differences in factors are described 
by the proportion of ratings that exceeded the midpoint of the 
survey scale which was labeled as “medium importance” and 
designated by the number “2” on the 0 to 4 rating scale. The three 
factors rated highest were depth of knowledge, availability, and 
attitude. A large proportion of ratings for each of these three 
factors were above the midpoint of the survey scale (94% > 2), and 
the mean rating was 3.5 for each of these factors. There seems to 
be little doubt among respondents that these factors are 
important in determining how well a SME will support 
curriculum development. Three other factors received high 
ratings: breadth of knowledge (88% > 2, M = 3.5), articulation 
skills (82% > 2, M = 3.4), and interpersonal skills (77% > 2, M = 
3.2). Two factors received relatively lower ratings: industry 
experience (65% > 2, M = 2.8) and cost (53% > 2, M = 2.5). 
Teaching experience received the lowest rating (12% > 2, M = 
1.7). 

The free-response section of the survey requested 
respondents to “enter any factors that you feel are important but 
were not part of this survey” and to describe each new factor 
entered. Seven factors were entered in this area of the survey and 
are reported in Table 2. One response was a duplicate of the 
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Table 1 

Ratings on Capability and Suitability Factors 


Factor 

* % > 2 

M 

SD 

Capability: Industry Experience 

65 

2.8 

0.66 

Capability: Teaching Experience 

12 

1.7 

0.86 

Capability: Breadth of Knowledge 

88 

3.5 

0.72 

Capability: Depth of Knowledge 

94 

3.5 

0.62 

Capability: Articulation Skills 

82 

3.4 

0.79 

Suitability: Availability 

94 

3.5 

0.62 

Suitability: Interpersonal Skills 

77 

3.2 

0.81 

Suitability: Attitude 

94 

3.5 

0.62 

Suitability: Cost 

53 

2.5 

1.10 


* Proportion of ratings higher than midpoint (“medium 


importance”) of the survey scale. 

capability factor of availability, and the other six were quite 
similar in meaning to one or more of the factors provided 
previously in the survey. 

Design of SME Scoring Instrument 

Considering the high ratings on most of the SME 
capability and suitability factors, it can be assumed that most of 
these characteristics are perceived to be important by 
instructional developers who have worked with SMEs. To identify 
and evaluate the strength of these factors in prospective SMEs, a 
SME scoring instrument was developed to help instructional 
developers select the most capable and suitable SMEs. The SME 
scoring instrument facilitates systematic aggregation of scores 
across factors. The additional guidance provided by SME scoring 
should reduce the risk and guesswork associated with choosing 
the best SMEs to support curriculum development. 
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Table 2 

Additional SME Factors Identified by Instructional Developer 
Respondents 


New Factor 

Description of New Factor 

Similarity to 

Survey Factor 

Editor & Reviewer 

Ability to accurately review 
materials produced by other 
team members 

Articulation Skills 

Follow Through 

Timely responses to requests 

Availability 

Adult Learning 

Appreciation of adult learning 
methodologies and practices 

Teaching Experience 

Availability 

Can provide training or be 
available upon request 

Availability 

Time 

Full-time personnel seem to 
work better than part-time 

Availability 

Currency 

Need folks that are up to date 
in the area in which they are SMEs 

Depth of Knowledge 

Dedication 

Need folks that are dedicated 
and willing to go above and beyond 

Attitude 


Three factors proposed earlier in this paper were 
eliminated from the SME scoring instrument due to their 
relatively low ratings on the survey. These included teaching 
experience, industry experience, and cost. The experience factors 
may also he limited in their utility, because technical knowledge 
and the capability to communicate technical information do not 
always have a direct correspondence to years of experience. 
Likewise, considerations of cost may not be stable indicators of 
SME merit since they are likely to vary significantly across 
projects and organizations that support curriculum and training 
development. 

To produce a profile for the evaluation of a potential SME, 
the SME scoring instrument was modeled after the “criteria of 
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merit checklist” (COMlist) proposed by Scriven (2000). To apply 
the COMlist, a SME’s merit must first be estimated using scores 
on three capability factors: breadth of knowledge, depth of 
knowledge, and articulation skills and three suitability factors: 
availability, interpersonal skills, and attitude. Merit scores range 
from one to 10 on each factor (10 = highest rating.) Potential 
SMEs can be assigned merit scores based on the instructional 
developer’s observations during an interview or during 
discussions of the curriculum development project or using 
additional sources of information. Eor example, scoring could be 
completed by more than one SME evaluator, or a supervisor 
(industry SME) or dean (faculty SME) may agree to score the 
factors for one or more SME candidates. 

To create a ranked profile for each SME candidate, the 
COMlist can be implemented with a spreadsheet that computes a 
product score for each factor. This score is the product of the 
SME’s merit score and the factor’s weight, assigned according to 
the instructional developer’s survey results reported above. Eor 
example, in the survey, 94% of the respondents rated the 
suitability factor “availability” above the midpoint of the 
importance scale, so the factor weight was set at 0.94. By 
assigning weights to each factor, the resulting product score 
incorporates the degree of the factor’s importance as determined 
by the perceptions of the survey respondents. Einally, when an 
overall percentage score is determined, factors with larger 
weights will influence the overall score to a greater degree than 
those with smaller weights. 

If more than one person is rated on the COMlist, 
individual factor scores and individual overall scores can either be 
aggregated or examined separately. The spreadsheet also 
generates a SME profile in the form of a column graph. The 
profile visually emphasizes strengths and weaknesses according 
to COMlist scores. 

SME COMlist Implementation 

Eigures 1 and 2 show an example of a completed COMlist 
and the resulting SME profile, respectively. To provide data for 
this example, the author evaluated a SME who was currently 
working on a curriculum development project that required 
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extensive knowledge of electronics (Mattoon & Frenzel, 2004). A 
weight for each factor was assigned according to the instructional 
developers’ survey results. Then the weighted score for each 
criterion was computed as a product of the score (1 to 10) entered 
hy the SME evaluator and the factor’s assigned weight. 

Figure 1 

COMlist Factor Product Scores and Overall Score for a Sample 
SME 


Criteria 

Score 

Weight 

Product 

Capability 




Breadth of Knowledge 

9 

0.88 

7.92 

Depth of Knowledge 

7 

0.94 

6.58 

Articulation skills 

6 

0.82 

4.92 

Suitability 




Availability 

9 

0.94 

8.46 

Interpersonal Skills 

8 

0.77 

6.16 

Attitude 

7 

0.94 

6.58 

1 Overall Score = 

80% 


Besides utilizing the capability and suitability factors 
identified in this study, the SME COMlist functions in some other 
useful ways. Like most checklists, the COMlist functions as a 
mnemonic device to prevent an evaluator from forgetting 
important characteristics to consider when selecting a SME. 
Eactors in the COMlist help stimulate the generation of useful 
questions when interviewing a SME. By providing a multiple set 
of criteria by which to judge a SME’s merit, the COMlist reduces 
the risk of a “halo effect,” the tendency to choose a SME who 
shows great promise in one or two areas but lacks other essential 
qualities. The COMlist is designed in a flexible format that can 
easily by altered by users. Eor example, the spreadsheet provides 
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for easy entry and deletion of factors and weights as additional 
data is collected (e.g., a larger instructional developers’ survey). 
Finally, the COMlist could facilitate more research on SME merit 
factors hy providing a vehicle for comparing SME scores to 
observations and measurements of actual performance on a 
curriculum development project. 

Figure 2 

Sample SME Profile Generated from COMlist Rating Data 


SME Profile 



Precautions for Implementing the SME COMlist 

The SME merit factors and example weights illustrated in 
Eigures 1 and 2 reflect data collected from the opinions of a small 
number of instructional developers. Consequently, the COMlist, 
in its present form, does not represent a statistically validated 
method and may not represent the larger population of 
instructional developers or the needs of all technical curriculum 
development projects. A more robust survey and additional study 
of the correspondence between SME scores and actual 
performance are needed to be sure of the instrument’s accuracy 
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and reliability. Additional research may prompt adjustment of the 
COMlist in several ways concerning inclusion of factors and 
assignment of weights to factors. 

Summary and Conclusions 

Subject matter experts play an essential role in technical 
curriculum development by providing accurate and up-to-date 
information that matches education, training, and workforce 
needs. The SME COMlist is proposed as a tool that enables 
instructional developers to evaluate a SME’s capability and 
suitability to support curriculum development. The COMlist 
provides a spreadsheet-based, single-entry scoring system that 
automatically calculates individual scores on six factors, provides 
an overall SME score, and generates a profile that illustrates a 
SME’s strengths and weaknesses. The value of the SME COMlist 
is based on its simplicity of design, ease of use, and usefulness in 
selecting SMEs who can competently support curriculum 
development. Choosing the most capable and suitable SMEs holds 
potential for significant gain in efficiency and effectiveness of 
curriculum development, which in turn, can improve the quality 
of technical education and training. 
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